Control apparatus for controlling an operation of at least one electronic device

ABSTRACT

According to the present invention, a device for controlling an operation of at least one electric device including a user interface unit having a conversation display window in which a conversation with the at least one electric device is displayed, a conversation input window configured to receive a character for controlling the operation of the at least one electric device, an icon selection window in which an icon corresponding to the at least one electric device is displayed, and a chatting screen on which a send button for transmitting the character received by the conversation input window is arranged, and a control unit configured to collectively control the operation of the at least one electric device based on a batch control instruction received by the conversation input window.

TECHNICAL FIELD

The present invention relates to a control device for monitoring andcontrolling home appliances.

BACKGROUND ART

Recently, along with the rapid development of wired/wireless datacommunication technology and the broad distribution of communicationnetworks, high-speed communication services such as the Internet havebeen widely popularized. In particular, as high-speed Internet servicehas been provided in households, conventional PC-based networktechnology has developed into home network technology for remotelycontrolling digital home appliances.

A home network system is a system for connecting digital homeappliances, sensors, light fixtures, heating and cooling devices, gasdevices, and so on over a wired or wireless local area network tocontrol appliances desired by a user anywhere at any time.

Such a home network system typically includes a home server (or a homemonitoring gateway) that functions as a master, a plurality of homenetwork devices (hereinafter referred to as control target devices) thatare monitored and controlled by the home server, and a controller thatlocally or remotely monitors and controls a plurality of control targetdevices through a connection with only the home server.

In this case, the home server is generally implemented on a residentialgateway (RG) or a wall pad, and a controller is generally implemented ina wall pad, a TV, a cell phone, a computer, etc. The home server and thecontroller may be integrated and implemented as one device. For example,a TV may serve as the control device in addition to the home server.

In the home network system, the home server may automatically recognizecontrol target devices, for example, a TV, a refrigerator, a lamp, amicrowave, a sensor, a boiler, and a washing machine, which areconnected to a home network through various physical media (e.g., apower line, a serial communication such as RS-232, Ethernet, etc.), andmay display the recognized home network devices on a screen to controldevices desired to be controlled by a user.

The controller monitors and controls a plurality of control targetdevices through communication with the home server.

In the home network system, the home server transmits data to orreceives data from an external network, which is mainly implemented asan Internet network.

The user uses a control device such as a PC, a notebook, a smartphone, atablet PC, a TV, a wearable device, and various remote controllers toaccess the home server located inside or outside the home and thenaccess a plurality of home appliances connected to the home networkinside the home.

In addition, when a plurality of electric devices are registered with anexternal server, the user accesses the plurality of electric devicesregistered with the external server via the external server,irrespective of the home server.

When the user uses the control device to monitor and control the controltarget devices inside the home, the use should select one of the controltarget devices, access the selected control target device over anetwork, perform authentication, and then monitor or control the controltarget device. Thus, it is difficult for the user to use the controldevice.

For example, home appliances such as an air conditioner or air cleanerfor conditioning an air and a humidifier for adjusting humidity areusually controlled together. In this case, the user shouldinconveniently and inefficiently issue an operation control command toeach electric device.

DISCLOSURE Technical Problem

One aspect provides a control device configured to input a predeterminedinstruction to a chatting screen to collectively control or individuallycontrol the operation of a plurality of electric devices, and a controlmethod thereof.

Another aspect provides a control device configured to perform aninstruction auto-complete function, a recommended instruction provisionfunction, a similar instruction provision function, and an iconinstruction provision function when an instruction is input to thechatting screen, and a control method thereof.

Still another aspect provides a control device having a notificationwindow for informing of an operation event of an electric device andconfigured to display a chatting unit screen for chatting with theelectric device when the notification window is touched as an input, anda control method thereof.

Yet another aspect provides a control device including an overview unithaving an icon of a home electric device, a speech bubble configured todisplay an operation state of the home electric device, and a movementbutton for laterally moving a screen on which an icon is displayed whenan app for a home network is executed, and a control method thereof.

Yet another aspect provides a control device including a master keyhaving a plurality of buttons for integrally controlling home electricdevices on the basis of a predetermined operation command when an appfor a home network is executed, and a control method thereof.

Yet another aspect provides a control device including a chatting unitconfigured to chat with a home electric device using a shortcut buttonfor which a chatting word is predetermined when an app for a homenetwork is executed, and a control method thereof.

Technical Solution

In accordance with an aspect, a control device for controlling anoperation of at least one electric device, the control device including:a user interface unit configured to have a chatting screen, the chattingscreen comprising a conversation display window for displaying aconversation with the at least one electric device, a conversation inputwindow for receiving a character for controlling an operation of the atleast one electric device, an icon selection window for displaying anicon corresponding to the at least one electric device, and a sendbutton for sending the character received through the conversation inputwindow; and a control unit configured to collectively control theoperation of the at least one electric device based on a batch controlinstruction received by the conversation input window.

The control device further includes a storage unit configured to store abatch control instruction for simultaneously controlling at least two ormore electric devices. When the input character is the batch controlinstruction, the control unit simultaneously controls the at least twoor more electric devices.

When the input character is the batch control instruction, the controlunit performs control such that an icon for all devices is displayed inthe icon selection window.

The control device further includes a storage unit configured to store acontrol instruction for individually controlling any one electricdevice. When the received character is the control instruction forindividually controlling any one electronic device, the control unitchecks an electric device corresponding to the control instruction,controls an operation of the checked electric device corresponding tothe control instruction, and performs control to display an icon of thechecked electric device in the icon selection window.

When the input character is the individual control instruction, thecontrol unit checks the electric device having the individual controlinstruction and performs control such that an icon of the checkedelectric device is displayed in the icon selection window.

When two or more electric devices correspond to the control instructionfor individually controlling any one electronic device, the control unitperforms control to display a list of the two or more electric devices.When any one electric device is selected, the control unit controls anoperation of the selected electric device based on the controlinstruction.

When the two or more electric devices correspond to the controlinstruction for individually controlling any one electronic device, thecontrol unit performs control to display an icon for all electricdevices in the icon selection window.

The control unit checks whether the character received by theconversation input window is a device name, checks an electric devicehaving the device name when the character received by the conversationinput window is the device name, and performs control such that an iconof the checked electric device is displayed in the icon selectionwindow.

When the device name and the character are received by the conversationinput window and then the send button is input, the control unit checkswhether the input character is a shortcut letter, checks an instructioncorresponding to the input character when the input character is theshortcut letter, and controls an operation of the checked electricdevice based on the checked instruction.

When the device name and the character are received by the conversationinput window and then the send button is input, the control unit checkswhether the input character is an instruction, checks an instructionhaving a similarity equal to or greater than a certain value with theinput character when the input character is not the instruction, andperforms control to display the checked instruction.

When the device name and the character are received by the conversationinput window, the control unit checks an instruction having the inputcharacter as a first character and performs control to display thechecked instruction.

The control unit checks an electric device having an icon selected onthe icon selection window, checks an operation state of the checkedelectric device, and performs control to display a list of executablerecommended instructions based on the checked operation state.

The control unit checks an electric device having the device namereceived by the conversation input window, checks an operation state ofthe checked electric device, and performs control to display a list ofexecutable recommended instructions based on the checked operationstate.

The control unit checks an electric device having the device namereceived by the conversation input window and performs control todisplay a list of instructions for controlling the checked electricdevice.

In accordance with another aspect, a device for controlling an operationof at least one electric device includes a user interface unit having aconversation display window in which a conversation with the at leastone electric device is displayed, a conversation input window configuredto receive a character for controlling the operation of the at least oneelectric device, an icon selection window in which an icon correspondingto the at least one electric device is displayed, and a chatting screenon which a send button for transmitting the character received by theconversation input window is arranged; and a control unit configured tocheck an instruction corresponding to the character received by theconversation input window and control the operation of the at least oneelectric device based on the checked instruction.

When a device name and the character are received by the conversationinput window and then the send button is input, the control unit checkswhether the input character is a shortcut letter and checks aninstruction corresponding to the input character when the inputcharacter is the shortcut letter.

When a device name and the character are received by the conversationinput window and then the send button is input, the control unit checkswhether the input character is an instruction, checks an instructionhaving a similarity equal to or greater than a certain value with theinput character when the input character is not the instruction, andperforms control to display the checked instruction.

When a device name and the character are received by the conversationinput window, the control unit checks an instruction having the inputcharacter as a first character and performs control to display thechecked instruction.

The control unit checks an electric device having an icon selected onthe icon selection window, checks an operation state of the checkedelectric device, and performs control to display a list of executablerecommended instructions based on the checked operation state.

The control unit checks an electric device having a device name receivedby the conversation input window, checks an operation state of thechecked electric device, and performs control to display a list ofexecutable recommended instructions based on the checked operationstate.

The control unit checks an electric device having a device name receivedby the conversation input window and performs control to display a listof instructions for controlling the checked electric device.

Advantageous Effects

According to an aspect, it is possible to easily control an operation ofa plurality of electric devices by inputting a predetermined instructionto a chatting screen to collectively control or individually control theoperation of the plurality of electric device.

It is also possible to easily input an instruction by providing aninstruction auto-complete function, a recommended instruction, a similarinstruction, and an icon instruction when an instruction is input to thechatting screen.

It is possible to allow an operation state of an electric device to beeasily checked by displaying a notification window for informing of anoperation event of an electric device in a quick panel window of thehome screen even when an app is not executed in the user interface unit.

In addition, it is possible to easily access a chatting screen, acontrol screen, and a service screen by touching a notification windowin the quick panel window as an input.

It is also possible to easily and quickly check an operation state of anelectric device since the operation state of the electric device isdisplayed through a pop-up window even when an app is not executed inthe user interface unit.

It is also possible to facilitate identification by displaying each of aplurality of home electric appliances and easily check an operationstate of the home electric appliance by displaying an operation state ofthe electric device in the vicinity of the icon as a speech bubble. Thatis, it is possible to provide convenience to the user.

In addition, since the operation state of the electric device may bechanged only by touching an icon, it is possible to easily control theoperation of the electric device and integrally control electric devicesat the same time using the plurality of buttons of the master key, thusenhancing convenience and efficiency of control of the electric device.It is also possible to rapidly and easily check the state information ofthe electric device by utilizing a pop-up window to output an event inwhich the state of the electric device is changed.

Furthermore, it is also possible to simply and easily transmit anoperation command to at least one electric device using a shortcutbutton of the chatting unit.

DESCRIPTION OF DRAWINGS

FIG. 1 is an exemplary diagram showing a home network system thatcommunicates with a control device according to an embodiment.

FIG. 2 is a block diagram showing a home network system thatcommunicates with a control device according to an embodiment.

FIG. 3 is an exemplary diagram showing a dashboard screen and a plug-inscreen of a control device according to an embodiment.

FIGS. 4 to 6 are exemplary diagrams showing an execution of an overviewunit in the dashboard screen of a control device according to anembodiment.

FIG. 7 is an exemplary diagram showing a master key in the dashboardscreen of a control device according to an embodiment.

FIG. 8 is an exemplary diagram showing an execution of the master key inthe dashboard screen of a control device according to an embodiment.

FIG. 9 is an exemplary diagram of a master key setting window (318) in asecond plug-in screen of a control device according to an embodiment.

FIGS. 10 and 11 are exemplary diagrams showing the setting of the masterkey in the second plug-in screen of a control device according to anembodiment.

FIGS. 12A to 18 are exemplary diagrams showing the execution of achatting unit in the dashboard screen of a control device according toan embodiment.

FIGS. 19 and 20 are exemplary diagrams showing a quick panel windowdisplayed on a home screen of a control device according to anembodiment.

FIG. 21 is a control block diagram showing a control device according toanother embodiment.

FIG. 22 is an exemplary diagram showing the display of a chatting screenof a control device according to another embodiment.

FIGS. 23 and 24 are exemplary diagrams showing the display of thechatting screen of a user interface unit provided in a control deviceaccording to another embodiment.

FIGS. 25 to 27 are exemplary diagrams showing the display of aconversation partner of the chatting screen of the user interface unitprovided in a control device according to another embodiment.

FIGS. 28 to 33 are exemplary diagrams showing an input of instructionsto the chatting screen of the user interface unit provided in a controldevice according to another embodiment.

MODES OF THE INVENTION

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings.

FIG. 1 is an exemplary diagram showing a home network system thatcommunicates with a control device according to an embodiment, and FIG.2 is a block diagram showing a home network system that communicateswith a control device according to an embodiment. The home networksystem that communicates with the control device includes a plurality ofelectric devices 100 a to 100 h and a network.

The electric devices 100 a to 110 h, each of which is a device thatperforms a predetermined unique function, transmit operation informationcorresponding to an operation state by performing an additional functionof communicating with the control device over the network, and performan operation corresponding to an operation command transmitted by thecontrol device. Here, the control device 300 includes a mobile device.

The electric devices include home appliances that are not portable, suchas a robot cleaner 100 a, a light fixture 100 b, a washing machine 100c, an air conditioner 100 d, a television 100 e, a cooking appliance 100f, a refrigerator 100 g, and a dishwasher 100 h, and may further includea terminal that is used in the home, such as a notebook, a PC, and atablet. In addition, the electric devices may further include a securitydevice in which a camera and a microphone are provided.

A network 200 is a home network that enables bidirectional communicationby connecting all electric or electronic devices used in the home to onesystem.

In detail, the network 200 may allow information to be shared betweenthe plurality of electric devices and the control device 300 over thewired/wireless Internet.

Home information transfer methods may be classified into a wired schemeand a wireless scheme. Examples of the wired scheme include a homephoneline networking alliance (PNA) using an existing telephone line,IEEE 1394, a power line, etc. Examples of the wireless scheme includehome radio frequency (RF), Bluetooth, Infrared Data Association (IrDA)and wireless LAN (e.g., WiFi), Zigbee, Z-wave, and a short-rangecommunication (e.g., NFC, etc.).

The network 200 includes a home server that manages and controls theplurality of electric devices.

The home server may be implemented on a residential gateway (RG), a wallpad, or a TV.

The home server may be implemented in software and provided in variouselectric devices or various mobile devices in the home. In addition thehome server may be implemented in hardware and combined with variouselectric devices or various mobile devices in the home.

That is, the home server displays operation states of the plurality ofelectric devices in the home, receives an operation command for at leastone of the plurality of electric devices from a user, and then transmitsthe operation command to the at least one electric device.

The control device 300 communicates with at least one of the pluralityof electric devices through wired or wireless communication.

In addition, the control device 300 may communicate with each of theplurality of electric devices in the home, and may transmit or receiveinformation of the plurality of electric devices through communicationwith the home server.

The control device 300 includes an app for communicating with the homenetwork to monitor and control the plurality of electric devices in thehome. When the app is executed by the user, the control device 300outputs an operation state of at least one of the plurality of electricdevices in the home, receives an operation command from the user, andcontrols an operation of the at least one electric device on the basisof the received operation command.

The control device 300, which is a terminal device movable to theoutside of the home such as a smartphone, a tablet PC, a notebook, etc.,or a device fixed inside the home such as a TV, various remotecontrollers, etc., may be a terminal device in which an app provided bya content provider may be installed, and may be a device having an app,software, or firmware built therein when the device is manufactured by amanufacturer.

As shown in FIG. 2, the control device 300 includes a user interfaceunit 310, a control unit 320, a storage unit 330, and a communicationunit 340.

The user interface unit 310 outputs operation states of pre-registeredelectric devices among the plurality electric devices disposed in thehome and receives an operation command for at least one of thepre-registered electric devices.

The user interface unit 310 includes a display unit for displaying theoperation states of the pre-registered electric devices and an inputunit for receiving the operation command from the user.

Here, the input unit is a touch panel for receiving an operationcommand, and the user interface unit 310 may be implemented as a touchscreen in which a display panel of the display unit and the touch panelof the input unit are integrally formed.

In addition, the input may be received by the input unit through anelectronic pen, a pointing device (mouse, etc.), and a keyboard inaddition to the touch panel. In this case, the input unit may be formedseparately from the display panel of the display unit. However, when theinput unit is a touch type electronic pen, the input unit may be formedintegrally therewith.

The user interface unit 310 includes a home screen on which icons forvarious apps are displayed.

The user interface unit 310 displays an icon of an app for monitoringand controlling an electric device on the home screen, transmits an appselection signal to the control unit 320 when the icon of the app isselected by the user, and displays a screen corresponding to executionof the app on the basis of a command of the control unit 320. The screendisplay configuration of the user interface unit will be describedbelow.

When the app is executed, the control unit 320 controls the userinterface unit 310 such that operation states of pre-registered electricdevices are output, and controls transmission of a signal correspondingto an operation command for at least one electric device on the basis ofan operation command input to the user interface unit 310.

The control configuration of the control unit 320 will be describedbelow in association with the screen display configuration of the userinterface unit.

The storage unit 330 stores identification information of home electricdevices registered by the user and an app for monitoring and controllingthe pre-registered electric devices.

The storage unit 330 stores an operation command for a pre-registeredelectric device for each button of a master key and stores an operationcommand for an electric device for each shortcut button of a chattingunit.

The communication unit 340 communicates with the pre-registered electricdevices.

Furthermore, the communication unit 340 may also communicate with thehome server.

The user interface unit 310 will be described in detail with referenceto FIGS. 3 to 20. In addition, a control operation of the control unit320 will also be described.

FIG. 3 is an exemplary diagram showing a dashboard screen and a plug-inscreen of a control device according to an embodiment.

When the app is executed, the user interface unit 310 displays adashboard screen 310 a as an initial screen.

Here, the dashboard screen 310 a is a screen on which basic informationregarding a plurality of electric devices connected through the homenetwork, information for controlling an operation, and so on aredisplayed.

When a sliding touch is input to the dashboard screen 310 a, the userinterface unit 310 switches the screen from the dashboard screen 310 ato a plug-in screen 310 b or 310 c.

That is, when a sliding touch applied from left to right is input to thedashboard screen 310 a, the user interface unit 310 displays a firstplug-in screen 310 b on which a device registration item for registeringan electric device is displayed. When a sliding touch applied from rightto left is input to the dashboard screen 310 a, the user interface unit310 displays a second plug-in screen 310 c on which a service item forservicing and setting the electric device is displayed.

In addition, when a sliding touch applied from right to left is inputwhile the first plug-in screen 310 b is displayed, the user interfaceunit 310 displays the dashboard screen 310 a.

According to the embodiment, the number of plug-in screens is two, butmay be one or three or more.

The control unit 320 checks a screen being displayed, performs switchingcontrol such that a screen positioned at the right side of the currentscreen is displayed when the sliding touch applied from right to left isinput, and performs switching control such that a screen positioned atthe left side of the current screen is displayed when the sliding touchapplied from left to right is input.

For example, when a sliding touch is input while the dashboard screen310 a is displayed, the control unit 320 checks a sliding touchdirection. The control unit 320 controls switching to the first plug-inscreen 310 b when the checked sliding touch direction is from left toright, and controls switching to the second plug-in screen 310 c whenthe checked sliding touch direction is from right to left.

The dashboard screen 310 a includes an overview unit 311 for displayinglist information and operation information of pre-registered electricdevices and widget units 312, 313, and 314 for displaying executioncommand information and monitoring information for rapidly and easilycontrolling operations of the pre-registered electric devices.

In addition, the dashboard screen 310 a further include a screenswitching unit 315 configured to directly switch from the dashboardscreen to the plug-in screen, directly switch from the plug-in screen tothe dashboard screen, or directly switch from one plug-in screen toanother plug-in screen.

The term “direct switching” includes switching the screen through asingle touch rather than the sliding touch.

That is, the screen switching unit 315 includes page buttons equal innumber to the plurality of screens. The arrangement of the plurality ofpage buttons corresponds to that of the plurality of screens. A pagebutton corresponding to the position of the currently displayed screenis displayed differently from the remaining page buttons. Thus, the usermay recognize the position of the currently displayed screen.

In addition, arrangement information of the plurality of screens may bepreset, and the arrangement of the screens may be changed by the user.

For example, the user interface unit according to the embodimentincludes a total of three screens, that is, the dashboard screen, thefirst plug-in screen, and the second plug-in screen. For the arrangementof the three screens, the first plug-in screen is disposed at the leftside of the dashboard screen, and the second plug-in screen is disposedat the right side of the dashboard screen.

Thus, there are a total of three page buttons of the screen switchingunit 315. Among the three page buttons, a left page button is linkedwith the first plug-in screen 310 b, a center page button is linked withthe dashboard screen 310 a, and a right page button is linked with thesecond plug-in screen 310 c.

When the left page button is touched, the first plug-in screen 310 b isdisplayed on the user interface unit. In this case, the left page buttonhas a different color from other buttons.

When the center page button is touched, the dashboard screen 310 a isdisplayed on the user interface unit. In this case, the center pagebutton has a different color from other buttons.

When the right page button is touched, the second plug-in screen 310 cis displayed on the user interface unit. In this case, the right pagebutton has a different color from other buttons.

That is, the control unit 320 performs control to check the page buttontouched among the plurality of page buttons of the screen switching unit315, check a screen corresponding to the checked page button, anddisplay the checked screen on the user interface unit 310.

The first plug-in screen 310 b and the second plug-in screen 310 c areeach a screen for changing setting information of a main function,setting an additional function, or receiving a service.

In detail, the first plug-in screen 310 b includes a device check buttonhaving registration information regarding a pre-registered electricdevice and a registration setting button 316 for registering a newdevice.

The second plug-in screen 310 c includes a setting change button forchanging settings of a master key, a chatting unit, and a home view unitof a widget unit, a service button for receiving a service for thepre-registered electric device, and a management button 317 for eachpre-registered electric device.

FIGS. 4 to 6 are exemplary diagrams showing an execution of an overviewunit in a dashboard screen of a control device according to anembodiment. The overview unit 311 will be described in detail withreference to FIGS. 4 to 6.

As shown in FIG. 4, the overview unit 311 displays a list ofpre-registered electric devices and displays icons A1 of thepre-registered electric devices.

In addition, the overview unit 311 further includes a movement button A2for moving a screen on which the icons of the pre-registered electricdevices are arranged.

That is, the movement button A2 is a button for moving the currentlydisplayed icon to change a display position of the icon in order todisplay icons of the remaining electric devices that are hidden and notdisplayed on one screen in the overview unit.

In this case, the movement button A2 is in a hidden state when thenumber of icons of the pre-registered electric devices is less than apredetermined number of icons that can be displayed on one screen. Themovement button A2 is in a displayed state when the number of icons ofthe pre-registered electric devices is greater than the predeterminednumber of icons that can be displayed on one screen.

The movement button A2 includes a first direction button < thatinstructs to move from a currently displayed screen to a screenextending in a first direction, and a second direction button > thatinstructs to move from the current displayed screen to a screenextending in a second direction. Here, the second direction is adirection opposite to the first direction.

When the first direction button < is selected, the overview unit 311shifts icons of a predetermined number of electric devices in thecurrent screen by one column in the second direction to rearrange theicons. When the second direction button > is selected, the overview unit311 shifts icons of a predetermined number of electric devices in thecurrent screen by one column in the first direction to rearrange theicons.

In this case, when the first direction button < is selected, theoverview unit 311 rearranges an icon of a new electric device at theleftmost column of the screen and removes an icon of an electric devicethat has been positioned at the rightmost column. When the seconddirection button > is selected, the overview unit 311 rearranges an iconof a new electric device at the rightmost column and removes an icon ofan electric device that has been positioned at the leftmost column.

In addition, when the first direction button < is selected, the overviewunit 311 displays a screen positioned at the left side of the currentscreen and displays an icon of an electric device arranged on the leftscreen. When the second direction button > is selected, the overviewunit 311 displays the screen positioned at the right side of the currentscreen and displays an icon of an electric device arranged on the rightscreen.

As shown in FIG. 4, when the second direction button > is touched whileicons of a cleaner, a light fixture, a washing machine, and an airconditioner are arranged on the current screen of the overview unit 311,icons of an air conditioner2, a robot cleaner2, and a televisionpositioned on the right screen of the current screen are displayed.

When the first direction button < is touched while the icons of the airconditioner2, the robot cleaner2, and the television are displayed, theoverview unit 311 moves the currently displayed screen to the leftscreen and displays the icons of the cleaner, the light fixture, thewashing machine, and the air conditioner positioned on the left screenagain.

Although the movement button of the overview unit 311 is touched, thecontrol unit 320 controls the user interface unit 310 such that screensof the widget units 312, 313, and 314 are maintained in the currentstate.

In addition, the overview unit 311 further includes a speech bubble A3that informs about an on/off state or a progressive state, whichindicates a current operation state, of a pre-registered electricdevice. The speech bubble A3 is positioned in the vicinity of the iconA1 such that the speech bubble A3 is easily recognized by the user.

The speech bubble A3 changes a notification indicating an operationstate of an electric device on the basis of a command of the controlunit 320.

That is, when any one of the icons displayed in the overview unit 311 istouched, the control unit 320 changes the operation state of theelectric device corresponding to the touched icon and controls the userinterface unit 310 such that the changed operation state is displayed ina speech bubble.

For example, when the icon of the air conditioner is touched while theair conditioner is in an on state, the control unit 320 controls the airconditioner to be powered off and controls the user interface unit 310such that the off state of the air conditioner is displayed in thespeech bubble. When the icon of the air conditioner is touched while theair conditioner is in an off state, the control unit 320 controls theair conditioner to be powered on and controls the user interface unit310 such that the on state of the air conditioner is displayed in thespeech bubble.

As shown in FIG. 5, on a condition that the icons of the robot cleaner,the light fixture, the washing machine, and the air conditioner aredisplayed, the overview unit 311 utilizes the speech bubble to informthat the operation of the air conditioner is powered off when the iconof the air conditioner that is powered on is touched, and utilizes thespeech bubble to inform that the operation of the robot cleaner iscleaning when the icon of the robot cleaner that is power off istouched.

In addition, when the icon of a washing machine is touched while thewashing machine is in an on state, the control unit 320 checks awork-in-progress state of the washing machine and controls the userinterface unit 310 such that the checked work-in-progress state of thewashing machine is displayed in the speech bubble.

As shown in FIG. 6, when an icon of the washing machine in an on stateis touched while the icons of the robot cleaner, the light fixture, thewashing machine, and the air conditioner are displayed, the overviewunit 311 utilizes the speech bubble to inform that a progress state ofthe washing machine is “one hour and ten minutes have passed.”

In addition, the control unit 320 may check at least one of an updatestate, an error state, and a reservation state of the pre-registeredelectric device, and may control the user interface unit such that thechecked state information is displayed in the speech bubble.

That is, the speech bubble A3 informs about any one of the updateinformation, the error information, and the reservation information ofthe electric device.

In addition, when an operation state of at least one electric device ischanged in response to a touch input to a master key 312 of the widgetunit and a chatting instruction input of a chatting unit 313, thecontrol unit 320 controls the user interface unit 310 such that thechanged operation state is displayed in a speech bubble.

The widget unit of the user interface unit 310 includes the master key312 for operating a pre-registered electric device on the basis of apreset operation command, the chatting unit 313 for chatting with atleast one of pre-registered electric devices, and a home view unit 314that outputs a home interior image. This will be described withreference to FIGS. 7 and 8.

FIG. 7 is an exemplary diagram showing a master key in a dashboardscreen of a control device according to an embodiment.

The master key 312 includes a button for integrally controllingoperations of a plurality of electric devices. This button previouslyincludes an operation command for each electric device.

The button of the master key 312 may be set on the basis of a user'sschedule.

As shown in FIG. 7, the master key 312 includes a going-out button B1, acoming-home button B2, and a good-night button B3.

When the master key 312 of the user interface unit is touched, thecontrol unit 320 checks the touched button to transmit a presetoperation command for each electric device corresponding to the checkedbutton to the pre-registered electric device, thus allowing thepre-registered electric device to operate on the basis of the presetoperation command. In this case, the user interface unit 310 informsabout the operation state of the pre-registered electric device througha speech bubble.

A button for integrally controlling the master key 312 may beadditionally added. The addition may be performed through a “master keysetting button” of the second plug-in screen.

In addition, the “master key setting button” may be arranged in themaster key.

In addition, an operation command for an electric device for each buttonof the master key 312 may be changed. This change may also be performedthrough the “master key setting button” of the second plug-in screen.

In addition, the button in which the master key 312 is set may bedeleted. This deletion may also be performed through the “master keysetting button” of the second plug-in screen.

FIG. 8 is an exemplary diagram showing an execution of a master key in adashboard screen of a control device according to an embodiment.

As shown in FIG. 8, the overview unit 311 displays icons of a robotcleaner, a light fixture, a washing machine, and an air conditioner anddisplays a speech bubble that informs about an operation state of eachelectric device in the vicinity of each of the icons.

It is assumed that the operation command for each electric devicecorresponding to the going-out button includes powering off the lightfixture and powering off the air conditioner, and the operation commandfor each electric device corresponding to the coming-home buttonincludes powering on the light fixture and powering on the airconditioner.

When the going-out button B1 of the master key 312 is touched (T) whilean operation state of the robot cleaner is off, an operation state ofthe light fixture is off, an operation state of the washing machine ison, and an operation state of the air conditioner is on, the userinterface unit 310 maintains the operation states of the robot cleanerand the light fixture in the off state, maintains the operation state ofthe washing machine in the on state in order to prevent the washingmachine from stopping while working, powers off the operation state ofthe air conditioner, and informs about the operation state of eachelectric device through a speech bubble.

When the coming-home button B2 of the master key 312 is touched (T)while the operation state of the robot cleaner is off, the operationstate of the light fixture is off, the operation state of the washingmachine is off, and the operation state of the air conditioner is off,the user interface unit 310 maintains the operation states of the robotcleaner and the washing machine in the off state, powers on the lightingdevice and the air conditioner, and informs about the operation state ofeach electric device through a speech bubble. In this case, when a usertouches (T) the icon of the air conditioner, the user interface unit 310powers off the air conditioner and informs about the power-off through aspeech bubble.

FIG. 9 is an exemplary diagram of a master key setting screen 318 in asecond plug-in screen of a control device according to an embodiment,and FIGS. 10 and 11 are exemplary diagrams showing the setting of amaster key in a second plug-in screen of a control device according toan embodiment.

When the master key setting button in the second plug-in screen istouched, the user interface unit 310 switches the dashboard screen intothe master key setting screen 318.

As shown in FIG. 9, the master key setting screen 318 includes an iconD1 of an pre-registered electric device, a movement button D2 for movinga list of icons of electric devices, a speech bubble D3 that informs anoperation command preset for each electric device, and a plurality ofpre-registered buttons D4, an editing button D5 of each of the pluralityof buttons, and an addition button D6 for adding an integral controlbutton.

As shown in FIG. 10, on a condition that the light fixture is poweredon, the television is powered off, the air conditioner is preset to atarget indoor temperature of 25° C., and the cleaner is preset to beoperating, when the going-out button is selected and the icon of thelight fixture is touched, the master key setting screen 318 changes theoperation command for the light fixture to an off state and then informsabout the change through a speech bubble. When the icon of the airconditioner is touched, the master key setting screen 318 changes theoperation command for the air conditioner to be an off state and theninforms about the change through a speech bubble.

Thus, the preset command for each electric device corresponding to thegoing-out button of the master key includes powering off the lightfixture, powering off the television, powering off the air conditioner,and operating the cleaner.

As shown in FIG. 11, on a condition that an going-out button, acoming-home button, a good-night button, and a good-morning button areset as integral control buttons, when an editing button of thegood-morning button is touched, the master key setting screen 318displays preset operation command information for each electric devicecorresponding to the good-morning button. In this case, when a deletionbutton of an option menu is touched, the master key setting screen 318displays a window that informs that all information associated with thegood-morning button is deleted. When an “OK” button is touched, themaster key setting screen 318 displays a plurality of buttons from whichthe good-morning button has been removed.

FIGS. 12A to 12D are an exemplary diagram of a chatting screen when achatting unit in a dashboard screen of a control device is executedaccording to an embodiment and exemplary diagrams illustrating thechange in a chatting screen corresponding to a conversation inputmethod.

When a chatting unit 313 is touched, the user interface unit 310switches a dashboard screen to a chatting screen 319. This is the sameas shown in portion (a) of FIG. 12.

As shown in portion (a) of FIG. 12, the chatting screen 319 includes aconversation display window E1 for displaying a conversation with atleast one electric device, an icon selection window E2 for selecting anicon for at least one electric device or an icon for all electricdevices, and a conversation input window E3 for selecting an inputmethod of a conversation with at least one of pre-registered electricdevices and inputting the conversation.

Here, the conversation input window E3 includes a text window E31 forselecting a message input method, a send button E32 for instructing tosend a text sentence, a list button E33 for instructing to display alist of shortcut buttons implying chatting instructions, a microphonebutton E34 for instructing to input voice, and a target window E35 fordisplaying an icon of an electric device with which the user chats.

When a touch is input, the text window E31 displays a QWERT type keypadE36.

In addition, the keypad E36 includes a button for switching to Koreancharacters or numbers. When the switching button is touched, Koreancharacters or numbers may be displayed.

Portions (b), (c), and (d) of FIG. 12 are exemplary diagramsillustrating changes in the chatting screen corresponding to aconversation input method. They will be described with reference toFIGS. 13 to 18.

As shown in portion (b) FIG. 12, when the text window is touched, thekeypad E36 is displayed, and thus a process for preparing a user to chatis performed.

As shown in FIG. 13, when the icon of the washing machine is touched,the chatting screen 319 displays the icon of the washing machine on thetarget window. When the text window is touched, the chatting screen 319displays the keypad E36, and thus performs a preparation process toallow the user to chat.

When a message for inquiring about a progress state of the washingmachine is input, and then the send button is touched, the chattingscreen 319 displays the input message on a representative displaywindow, receives information regarding a current progress state from thewashing machine, and displays the received progress state on a progressbar, etc.

In addition, the chatting screen displays a time at which the chattingword is input to the conversation display window together.

In addition, as shown in FIG. 14, when the icon of the air conditioneris touched, the chatting screen 319 displays the icon of the airconditioner on the target window. When the text window is touched, thechatting screen 319 displays the keypad E36, and thus performs apreparation process to allow the user to chat.

When an operation command is input through a message, and then the sendbutton is touched, the chatting screen 319 displays the inputon-operation command on the conversation display window and displays acurrent indoor temperature provided by the air conditioner. When atarget indoor temperature is input through a message, and then the sendbutton is touched, the chatting screen 319 displays the input targetindoor temperature on the conversation display window, receives aresponse that the command will be performed from the air conditioner,and displays the received response on the conversation display window.

As shown in FIG. 15, when an icon for selecting all devices is selected,the chatting screen 319 displays the icon for selecting all devices onthe target window. When the text window is touched, the chatting screen319 displays the keypad E36 and thus performs a preparation process toallow the user to chat.

When a chatting word for monitoring all devices is input through akeypad, the chatting screen 319 displays the input chatting word on theconversation display window. When monitoring information is receivedfrom a pre-registered electric device, the chatting screen 319 displaysthe received monitoring information for each electric device on theconversation display window.

When the conversation display window is touched while the monitoringinformation for each electric device is displayed, the chatting screen319 hides the keypad.

As shown in FIG. 16, when an icon for selecting all devices is selected,the chatting screen 319 displays the icon for selecting all devices onthe target window. When the text window is touched, the chatting screen319 displays the keypad E36 and thus performs a preparation process toallow the user to chat.

When a chatting word for informing all devices of the going-out is inputthrough the keypad, the chatting screen 319 displays the input chattingword on the conversation display window. When operation stateinformation corresponding to the going-out is received from apre-registered electric device, the chatting screen 319 displays thereceived operation state information for each electric device on theconversation display window.

When the conversation display window is touched while the operationstate information for each electric device is displayed, the chattingscreen 319 hides the keypad.

When the icon of the air conditioner is touched, the chatting screen 319displays the icon of the air conditioner on the target window. When thetext window is touched, the chatting screen 319 displays the keypad E36and thus performs a preparation process to allow the user to chat.

When an operation command is input through a message, and then the sendbutton is touched, the chatting screen 319 displays the inputon-operation command on the conversation display window and displays acurrent indoor temperature provided by the air conditioner. When atarget indoor temperature is input through a message, and then the sendbutton is touched, the chatting screen 319 displays the input targetindoor temperature on the conversation display window, receives aresponse that the command will be performed from the air conditioner,and displays the received response on the conversation display window.

As shown in portion (c) of FIG. 12, when the list button E33 is touched,the chatting screen 319 displays a plurality of shortcut buttons E37 onwhich respective chatting instructions are displayed and thus performs apreparation process to allow the user to chat.

FIG. 17 shows an example of a shortcut button for each pre-registeredelectric device.

Portion (a) of FIG. 17 is an exemplary diagram of a shortcut button listcorresponding to the icon for selecting all devices. When the icon forselecting all devices is selected, a plurality of shortcut buttons haverespective operation commands for integrally controlling all of thepre-registered electric devices. In addition, the plurality of shortcutbuttons for sending the control command to all devices correspond tobuttons of the master key.

Portion (b) of FIG. 17 is an exemplary diagram of a shortcut button listcorresponding to the operation control of the air conditioner, portion(c) of FIG. 17 is an exemplary diagram of a shortcut button listcorresponding to the operation control of the robot cleaner, portion (d)of FIG. 17 is an exemplary diagram of a shortcut button listcorresponding to the operation control of the washing machine, portion(e) of FIG. 17 is an exemplary diagram of a shortcut button listcorresponding to the operation control of the cooking appliance, andportion (f) of FIG. 17 is an exemplary diagram of a shortcut button listcorresponding to the operation control of the refrigerator.

As shown in FIG. 18, when the icon of the air conditioner is touched,the chatting screen 319 displays the icon of the air conditioner on thetarget window. When information regarding a current indoor temperatureis received from the air conditioner, the chatting screen 319 displaysthe received current indoor temperature on the conversation displaywindow.

When the list button is touched, the chatting screen 319 displays theplurality of shortcut buttons E37 on which the chatting instructions foroperation control associated with the air conditioner are displayed.

When a shortcut button for instructing air-conditioning is touched amongthe plurality of shortcut buttons, the chatting screen 319 displays aconversation corresponding to the touched shortcut button on theconversation display window, receives an operation state from the airconditioner, and displays a conversation of “an air conditioningoperation will be performed.” When the list button is touched after apredetermined period of time has passed, the chatting screen 319 hidesthe list button. Subsequently, when the list button in the conversationinput window is touched, the chatting screen 319 displays the iconselection window and the shortcut button list.

When the icon for selecting all devices is touched while the iconselection window and the shortcut button list are displayed, thechatting screen 319 displays the icon for selecting all devices on thetarget window and displays a list of shortcut buttons corresponding tothe icon for selecting all devices.

Subsequently, when the going-out button is touched among the pluralityof shortcut buttons, the chatting screen 319 displays a conversationinforming that the going-out button has been touched on the conversationdisplay window and sends a preset operation command for each electricdevice corresponding to the going-out button to each electric device.

In addition, the chatting screen displays a time at which the chattingword is input to the conversation display window together.

As shown in portion (d) of FIG. 12, when the microphone button E34 istouched, the chatting screen 319 displays a voice input window E38 forreceiving a voice command and thus performs a preparation process toallow the user to chat.

As shown in portion (d) of FIG. 12, when the icon of the washing machineis touched, the chatting screen 319 displays the icon of the washingmachine on the target window. When the microphone button E34 is touched,the chatting screen 319 displays a voice input window E38 for inputtinga voice command and thus performs a preparation process to allow theuser to chat.

In addition, the voice input window includes a button for setting avoice command feedback, a button for inputting a voice, and a keypadswitching button for performing switching to the keypad.

When a word is uttered while the button for inputting a voice istouched, the chatting screen 319 collects and then recognizes the voicethrough a microphone, and displays the recognized voice on theconversation display window. When a query corresponding to therecognized voice is about a progress state of the washing machine, thechatting screen 319 receives information regarding a current progressstate from the washing machine and displays the received progress stateusing a progress bar, etc.

In addition, the chatting screen displays a time at which the chattingword is input to the conversation display window together.

The home view unit 314 may output interior images of the user's home atpredetermined intervals. When a touch command is input, the home viewunit 314 may expand a home view image and output the expanded image tothe dashboard screen.

In addition, the home view unit 314 may output a previous image inresponse to the user's command.

FIGS. 19 and 20 are exemplary diagrams showing a quick panel windowdisplayed on a home screen of a control device according to anembodiment.

As shown in FIG. 19, the control device determines whether an operationstate of at least one electric device has been changed while an app isnot executed. When it is determined that the operation state of the atleast one of electric device has been changed, the control devicedisplays notification information corresponding to the change in theoperation state on a home screen of the user interface unit 310 througha quick panel window.

In addition, while the app is not executed, the control device displaysnotification information about an update, error, and management periodof the at least one electric device on the home screen through the quickpanel window.

When the notification information in the quick panel window is touched,the user interface unit 310 displays in detail information about acorresponding electric device.

As shown in FIG. 20, the user interface unit may display a tutorialabout the corresponding electric device and perform association with anoperation for managing the washing machine.

For example, when a button for instructing to clean a laundry drum istouched while a message such as “clean laundry drum of washing machine”is displayed, the user interface unit sends a laundry drum cleaningcommand to the washing machine and displays a cleaning-in-progress stateof the laundry drum on the user interface unit.

FIG. 21 is a control block diagram of a control device according toanother embodiment, which will be described with reference to FIG. 22.

As shown in FIG. 21, a control device 300 communicates with anothercontrol device, and sends a control command to at least one of aplurality of electric devices on the basis of an operation command orinformation associated with the operation command sent by the othercontrol device.

Here, the other control device is an information technology (IT) devicethat may be installed in a human body, and may be a wearable deviceprovided as, e.g., a wrist watch, a belt, or glasses. The wearabledevice is capable of input/output and communication.

For example, the control device 300 may be a smartphone thatcommunicates with a wrist-watch-type device, and the other controldevice may be a wrist-watch-type device.

Alternatively, the control device 300 may be a TV associated with a TVremote controller, and the other control device may be a separate TVremote controller.

That is, the other control device receives an operation command for atleast one of a plurality of electric devices from a user and transmitsthe received operation command to a control device 300 which ispre-registered. An app for controlling an electric device in the homemay be installed in the other control device.

Although not shown, the other control device may include an input unit.

Examples of an input through the input unit of the other control devicemay include a touch input, an electronic pen, a pointing device (e.g., amouse, etc.), a keyboard, a voice, and an operation.

In addition, the other control device may include an output unit.

Examples of an output through the output unit of the other controldevice may include an output through a display unit such as a touchpanel, etc., a voice output, a vibration output, and a light output.

The control device 300 and the other control device may be connected bywire or wirelessly. The wired connection scheme may include MobileHigh-Definition Link (MHL) and Universal Serial Bus (USB), and thewireless connection scheme may include Bluetooth, WiFi, Zigbee, IrDA,and short-range communication.

Although not shown, a method of controlling an electric device using thecontrol device 300 and the other control device will be described, forexample, as follows.

The input through the other control device may be a voice input. Whenthe other control device has a touch panel, the input may be a touchinput.

On a condition that a control command is input to the other controldevice through the voice input, when the other control device transfersvoice information to the control device, the control device 300 analyzesthe voice, acquires a corresponding instruction, and performs a controloperation corresponding to the instruction.

Alternatively, when the other control device digitalizes or encodesvoice information and transfers the digitalized or encoded voiceinformation to the control device 300, the control device 300 analyzesthe digitalized or encoded information, acquires a correspondinginstruction, and performs a control operation corresponding to theinstruction.

Alternatively, when the other control device analyzes voice informationand transfers specific information indicating a result of the analysisto the control device 300, the control device 300 acquires a commandassociated with the specific information and performs a controloperation corresponding to the instruction.

Alternatively, when the other control device analyzes voice information,acquires an instruction corresponding to the voice information, andtransmits the acquired instruction to the control device 300, thecontrol device 300 performs a control operation corresponding to theinstruction.

On a condition that a control command is input to the other controldevice through the touch input, when the other control device transfersinformation corresponding to a touched icon to the control device 300,the control device 300 acquires an instruction corresponding to theinformation and performs a control operation corresponding to theinstruction.

Alternatively, when the other control device acquires the correspondingcommand through the information corresponding to the touched icon andtransmits the acquired instruction to the control device 300, thecontrol device 300 performs a control operation corresponding to theinstruction.

Here, the control command may be a command for controlling a pluralityof electric devices.

When the command for controlling the plurality of electric devices isinput to the other control device, the other control device determineswhether the input command is the command for controlling the pluralityof electric devices and transmits a result of the determination to thecontrol device 300.

The determination result may be transmitted simultaneously with, orseparately from, the input command.

Alternatively, the other control device determines whether the inputcommand is the command for controlling the plurality of electricdevices. When the input command is a command for controlling theplurality of electric devices, the other control device directlytransmits a control command to electric devices to be controlled,instead of through the control device 300.

Alternatively, when the command for controlling the plurality ofelectric devices is input to the other control device, the controldevice 300 determines whether the command for controlling the pluralityof electric devices is valid and performs a corresponding controloperation.

When using the other control device as described above, the user maysimply control a home electric device through a wrist-watch-type deviceor a TV remote controller.

For example, the user may collectively power off home appliances bysaying, to the wrist-watch-type device, “Good night.”

A control device according to another embodiment has a differentchatting configuration from the control device according to anembodiment.

As shown in FIG. 21, the control device 300 includes a user interfaceunit 600, a control unit 370, a storage unit 330, and a communicationunit 340.

The user interface unit 600 outputs operation states of pre-registeredelectric devices among the plurality electric devices disposed in thehome and receives an operation command for at least one of thepre-registered electric devices.

The user interface unit 600 includes a display unit for displaying theoperation states of the pre-registered electric devices and an inputunit for receiving the operation command from the user.

Here, the input unit is a touch panel for receiving an operationcommand, and the user interface unit 600 may be implemented as a touchscreen into by integrally forming a display panel of the display unitand the touch panel of the input unit.

In addition, the input of the input unit may be received through anelectronic pen, a pointing device (mouse, etc.), a keyboard, voicerecognition, and gesture recognition in addition to the touch panel.

In this case, the input unit may be formed separately from the displaypanel of the display unit. However, when the input unit is a touch typeelectronic pen, the input unit may be formed integrally therewith.

The user interface unit 600 displays an icon of an app for monitoringand controlling an electric device on the home screen, transmits an appselection signal to the control unit 370 when the icon of the app isselected by the user, and displays a screen corresponding to anexecution of the app on the basis of a command of the control unit 370.

When a chatting unit displayed on the home screen of the user interfaceunit 600 is touched, the home screen is switched to a chatting screen.

When a chatting unit of a dashboard screen is touched, the userinterface unit 600 switches the dashboard screen to a chatting screen600 a and displays the chatting screen 600 a. This is the same as shownin portion (a) of FIG. 22.

As shown in portion (a) of FIG. 22, the chatting screen 600 a includes aconversation display window 610 for displaying a conversation with atleast one electric device, a conversation input window 620 for selectingan input method of a conversation with at least one of pre-registeredelectric devices and inputting the conversation, an icon selectionwindow 630 for selecting an icon for at least one electric device whoseoperation is to be monitored and controlled or an icon for all electricdevices, and a send button 640 for transferring text.

As shown in portion (b) of FIG. 22, when all electric devices displayedin the icon selection window are selected or when a home icon forcontrolling a plurality of electric devices is selected, the chattingscreen 600 a of the user interface unit 600 additionally outputs atarget window 631 for displaying a list of the pre-registered electricdevices and a shortcut word button window 632 for displaying apredetermined instruction.

In addition, the target window 631 and the shortcut word button window632 or the shortcut word button window 632 other than the target window631 may be output as a text window 650.

As shown in portion (c) of FIG. 22, when the conversation input window620 is selected, the chatting screen 600 a of the user interface unit600 additionally outputs the text window 650 for selecting a characterinput scheme.

Here, when the touch is input, the text window 650 includes a QWERT typekeypad or a 10-key type keypad.

In addition, the keypad includes a button for switching to Koreancharacters or numbers. When the switching button is touched, Koreancharacters or numbers may be displayed.

Alternatively, the user interface unit may further display a microphonebutton that instructs a voice input.

When the conversation display window 610 is touched, the chatting screen600 a of the user interface unit switches the target window, theshortcut word button window, or the text window to a hidden state.

When the app is executed, the control unit 370 controls the userinterface unit 600 such that operation states of pre-registered electricdevices or notification information needed for the user is output, andcontrols transmission of a signal corresponding to an operation commandfor at least one electric device on the basis of an operation commandinput to the user interface unit 600 by touch.

When a batch control instruction is input, the control unit 370 switchesa conversation partner device to all devices.

Here, the batch control instruction is not an instruction forcontrolling one electric device but an instruction predefined to controla plurality of electric devices.

The control unit 370 performs control to transmit individual controlinstructions associated with a batch control or a batch controlinstruction itself to a plurality of electric devices associated withthe batch control instruction.

In addition, the control unit 370 performs control to transmit the batchcontrol instruction to all electric devices in the home.

Here, the individual control instruction is an instruction to beperformed by a corresponding electric device in association with thebatch control instruction.

When an individual control instruction corresponding to any one electricdevice is input, the control unit 370 automatically switches thecorresponding electric device to a conversation partner device. When thedevice name of an electric device is input, the conversation partnerdevice is switched to an electric device having the input device name.

The control unit 370 may receive a selection of the batch controlinstruction or the individual control instruction through the shortcutword button window rather than the text window.

Such a configuration of the control unit will be described below indetail on the basis of an example of an input/output of a chattingscreen.

The storage unit 330 stores identification information of home electricdevices registered by the user and an app for monitoring and controllingthe pre-registered electric devices and stores the batch controlinstruction and the individual control instructions.

The storage unit 330 stores similar control instructions and recommendedinstructions for each character and stores an operation command for eachinstruction.

The communication unit 340 communicates with an electric device and ahome server in the home.

An example of displaying chatting of the chatting screen of the userinterface unit will be described in detail with reference to FIGS. 23 to27.

FIGS. 23 and 24 are exemplary diagrams of a chatting screen that is usedto collectively control a plurality of electric devices.

As shown in FIG. 23, when the icon selection window 630 is touched, theuser interface unit displays an icon (e.g., a house shaped icon) of apre-registered electric device. When an icon for all devices is touchedamong icons of the pre-registered electric devices is touched, the userinterface unit displays the icon for all devices on the icon selectionwindow 630.

When the batch control instruction for the plurality of electric devicesis input through the conversation input window 620 of the user interfaceunit, the control unit transmits individual control instructionsassociated with a batch control (that is, instructions to be performedby a corresponding device in association with the batch controlinstruction) or the batch control instruction itself to a plurality ofelectric devices associated with the batch control instruction. Inaddition, the control unit transmits the batch control instruction toall electric devices in the home.

When the batch control instruction is input to the conversation inputwindow 620 of the user interface unit together with a word (e.g., a homeor a nickname thereof) referring to all devices, the control device maytransmit individual control instructions associated with a batch control(that is, instructions to be performed by a corresponding device inassociation with the batch control instruction) or the batch controlinstruction itself to a plurality of electric devices associated withthe batch control instruction.

In addition, the control unit may transmit the batch control instructionto all electric devices in the home. In this case, the batch control canbe easily performed irrespective of who was a conversation partnerbefore the batch control instruction is input.

After the batch control instruction is transmitted, the control unitallows information about the result to be displayed on the conversationdisplay window 610 of the chatting screen of the user interface unit.

As shown in drawings, each device associated with the batch controlinstruction may display an operation state or a result of the operation.Although not shown, conversation partners indicating all devices maycollectively display their states or results.

For example, for a result of a batch control instruction “Going out,”each associated devices (i.e., an air conditioner, a robot cleaner,etc.) displays the results. That is, on the assumption that, when theinstruction “Going out” is input, the air conditioner is set to bepowered off, and the cleaner is set to perform a cleaning operation, thecontrol device displays a conversation that the power is off on theconversation display window of the chatting screen together with theicon of the air conditioner displays a conversation that the cleaningoperation is started together with the icon of the cleaner.

In addition, a conversation partner having a house shaped icon maycollectively display the state or result (For example, “Going out modehas been executed,” etc.).

Although not shown in FIG. 23, when the batch control instruction for aplurality of predefined devices is input to the conversation inputwindow 620, the control device may also switch the conversation partnerdevice to all devices and transmit individual control instructionsassociated with a batch control (that is, instructions to be performedby a corresponding device in association with the batch controlinstruction) or a batch control instruction itself to a plurality ofelectric devices associated with the batch control instruction. Inaddition, the control unit can transmit the batch control command to allelectric devices in the home.

As shown in FIG. 24, when the batch control instruction is input to theconversation input window 620 while the icon for all devices is notselected on the icon selection window 630 of the chatting screen, theuser interface unit automatically changes an electric device icondisplayed in the icon selection window 630 to the icon for all devicesand displays the icon for all devices.

For example, when a “Going out” instruction, which is a batch controlinstruction, is input while an icon of a robot cleaner is displayed inthe icon selection window 630 of the chatting screen, the user interfaceunit displays the icon for all devices on the icon selection window 630and displays the batch control instruction “Going out” on theconversation display window 610 in addition to the icon for all devices.

Furthermore, the control unit of the control device determines whetherthe going-out instruction input to the user interface unit is the batchcontrol instruction.

FIGS. 25, 26, and 27 are exemplary diagrams of a chatting screen that isused to individually control any one electric device.

As shown in FIG. 25, when the individual control instruction is inputwhile the icon for all devices is displayed in the icon selection window630 of the chatting screen, the user interface unit displays an icon ofan electric device corresponding to the input individual controlinstruction on the icon selection window 630 and displays the inputindividual control instruction on the conversation display window 610.

In this case, the control unit of the control device determines whetherthe input instruction is the individual control instruction, checks anelectric device having the determined individual control instruction,and transmits an operation command corresponding to the individualcontrol instruction to the checked electric device, and displaysinformation about an operation state of the checked electric device onthe conversation display window 610 of the chatting screen of the userinterface unit.

For example, when a “Recharging” instruction is input as the individualcontrol instruction while an icon for all devices is displayed in theicon selection window 630 of the chatting screen, the user interfaceunit displays the icon of the robot cleaner on the icon selection window630.

In this case, the control unit of the control device checks an electricdevice that performs recharging, which is the input instruction. Whenthe checked electric device is determined as the robot cleaner, thecontrol unit transmits a recharging command to the robot cleaner,receives information about a recharging operation from the robotcleaner, and displays the received information on the conversationdisplay window 610 of the chatting screen of the user interface unit.

As shown in FIG. 26, when a message is input through the text window 650while the icon for all devices is displayed in the icon selection window630, the user interface unit displays the input message on theconversation input window 620. When a send button 640 is touched afterthe message displayed on the conversation input window 620 is input, theuser interface unit displays the input message on the conversationdisplay window 610.

The user interface unit displays at least one of a list of icons of aplurality of electric devices having the same instruction and a list ofdevice names of the plurality of electric devices. When any one istouched from the list, the user interface unit displays, on theconversation input window 620, a device name of the electric devicedisplayed in the touched region.

In addition, the user interface unit can display the icon list and thedevice names of the plurality of electric devices displayed on theconversation display window 610 and receive a device name of an electricdevice through the conversation input window 620.

That is, the control unit of the control device checks whether aninstruction, that is, a message displayed on the conversation inputwindow 620 is a duplicated instruction corresponding to the plurality ofelectric devices. When the instruction is determined to be duplicated,the control unit displays a list of icons of the plurality of electricdevices associated with the duplicated instruction on the conversationdisplay window 610 of the chatting screen of the user interface unit.

When at least one icon is selected from the list of icons of theplurality of electric devices displayed on the conversation displaywindow 610 of the user interface unit, the control unit of the controldevice checks an electric device corresponding to the selected at leastone icon and displays the device name of the checked electric device onthe conversation input window 620.

Furthermore, the control device stores information about electricdevices for each duplicated instruction and stores an operation commandfor each electric device.

For example, when a “Turn off” instruction is input as the duplicatedinstruction while an icon for all devices is displayed in the iconselection window 630 of the chatting screen, the user interface unitchecks electric devices to which the “Turn off” instruction may beapplied, that is, electric devices being turned on and displays a listof icons or device names of the checked electric devices.

The user interface unit receives a touch on an icon of at least oneelectric device from the list of icons of the electric devices orreceives the device name through the conversation input window andtransmits an “Turn off” operation command to the received electricdevice.

As shown in FIG. 27, when a name of a specific electric device (or aspecific input for identifying an electric device) is input through thetext window 650, the conversation state is changed to the correspondingelectric device irrespective of the current conversation partner.

For example, the user interface unit checks whether the messagedisplayed on the conversation input window 620 is the device name. Whenthe checked message is determined to be the device name, the userinterface unit checks an electric device having the determined devicename and displays an icon of the checked electric device on the iconselection window 630.

When a message is input through the text window 650, the user interfaceunit displays the input message on the conversation input window 620.When a send button 640 is touched after the message displayed on theconversation input window 620 is input, the user interface unit displaysthe input message on the conversation display window 610, checks theinput instruction on the conversation input window 620, and displays thechecked “Recharging” instruction on the conversation display window 610.

For example, when “recharge robot cleaner” is input, the conversationpartner is changed to the robot cleaner although previous conversationpartners are all devices (a house shaped icon), a control command torecharge is transmitted to the robot cleaner. In addition, theconversation partner icon is changed to the robot cleaner.

The control unit of the control device transmits an operation commandcorresponding to the checked “Recharging” instruction to the robotcleaner, receives information about a recharging operation from therobot cleaner, and allows the received information to be displayed onthe conversation display window 610 of the user interface unit.

An example of inputting an instruction of the chatting screen of theuser interface unit will be described in detail with reference to FIGS.28 to 33.

As shown in portion (a) of FIG. 28, the user interface unit directlyreceives natural language the user uses through the conversation inputwindow of the chatting screen. As shown in portion (b), the userinterface unit selects an instruction preset for each operation commandand receives the selection through the conversation input window of thechatting screen. As shown in portion (c), the user interface unitdisplays an icon target window for each electric device and a shortcutword button window 632 for which an instruction for each electric deviceis set on the chatting screen and receives a touch on an icon and abutton or receives an input through voice.

When the input is received through voice, the control device may displaythe recognized voice on the conversation window. In addition, when theinput is received through voice, the control device may display a groupof instruction candidates for the recognized voice and the group may bedisplayed such that the user selects an appropriate instruction from thegroup.

As shown in FIG. 29, when the message displayed on the conversationinput window 620 is the device name, the user interface unit displays anicon of an electric device having the device name on the icon selectionwindow 630. When the conversation input window 620 is touched in themanner of a tap or a long-tap, the user interface unit displays thepreset instruction list. In this case, the instructions are set as alist for each electric device.

When any one instruction is touched from the preset instruction list,the user interface unit displays the touched instruction on theconversation input window 620. When the send button is touched by auser, the user interface unit displays a devices name and an instructionof an electric device that are input to the conversation display window610.

The control unit of the control device checks whether the messagedisplayed on the conversation input window 620 is the device name. Whenit is determined that the checked message is the device name, thecontrol unit checks the electric device having the determined devicename and displays an icon of the checked electric device in the iconselection window 630.

For example, when “Robot Cleaner” is input to the conversation inputwindow 620, the user interface unit displays an icon of the robotcleaner on the icon selection window 630. When the conversation inputwindow 620 is touched as a tap or long-tap, the user interface unitdisplays a preset instruction list for controlling the robot cleaner.

When Turn-off is touched from the preset instruction list, the userinterface unit displays the touched instruction on the conversationinput window 620. When the send button is touched, the user interfaceunit displays a conversation selected by the user on the conversationdisplay window 610.

As shown in FIG. 30, when the message displayed on the conversationinput window 620 is the device name, the user interface unit displays anicon of an electric device having the device name on the icon selectionwindow 630 and displays a list of instructions that begin with at leastone character input through the text window 650. That is, when the atleast one character is input, the user interface unit displays aninstruction based on an automatic completion function.

When any one instruction is selected from the instruction list, the userinterface unit displays the selected instruction on the conversationinput window 620. When the send button is touched, the user interfaceunit displays the device names and instructions on the conversationdisplay window.

Furthermore, the control device stores an instruction and orderinformation of words composing of the instruction.

For example, when “Robot Cleaner” is input through the text window 650,the user interface unit displays a robot cleaner on the conversationinput window 620 and also displays an icon of the robot cleaner on theicon selection window 630. When “s” is input through the text window650, the user interface unit displays a list of instructions that beginwith “s.”

When “start” is selected from among the instructions that begin with“s,” the user interface unit displays “start” on the conversation inputwindow 620. When the send button is touched, the user interface unitdisplays the device name (Robot Cleaner) and the instruction (start) onthe conversation display window 610.

As shown in FIG. 31, when the message displayed on the conversationinput window 620 is the device name, the user interface unit displays anicon of an electric device having the device name on the icon selectionwindow 630 and displays an instruction input through the text window 650on the conversation input window. When the send button is touched, theuser interface unit displays the device name and the instruction on theconversation display window.

When the input instruction is an unrecognized instruction, the userinterface unit displays an instruction that is most similar to the inputinstruction as a selectable button. When the displayed button isselected, the selected instruction is displayed again along with thedevice name.

In this case, the control unit of the control device determines asimilarity of words between the input instruction and instructions ofthe electric device having the input device name, determines aninstruction having a similarity equal to or greater than a certainvalue, and selects the determined instruction as a similar instruction.

For example, when “Robot Cleaner” is input through the text window 650,the user interface unit displays a robot cleaner on the conversationinput window 620 and also displays an icon of the robot cleaner on theicon selection window 630. When “star” is input through the text window650, the user interface unit displays “start,” which is a command thatis the most similar to “star.”

That is, the control unit of the control device compares words composingof an instruction for controlling the robot cleaner with ‘star’ anddetermines “start,” which is a command that is the most similar to“star,” as the similar instruction.

As shown in FIG. 32, when the message displayed on the conversationinput window 620 is the device name, the user interface unit displays anicon of an electric device having the device name on the icon selectionwindow 630 and displays, as a recommended instruction list, aninstruction based on the operation state of the electric device havingthe device name. When any one instruction is selected from therecommended instruction list, and then the send button is touched, theuser interface unit displays the device name and the instruction on theconversation display window 610.

In this case, the control unit of the control device checks theoperation state of the electric device having the input device name,checks an executable operation command on the basis of the checkedoperation state, and checks an instruction corresponding to the checkedoperation command.

For example, when “Robot Cleaner” is input through the text window 650,the user interface unit displays a robot cleaner on the conversationinput window 620 and also displays an icon of the robot cleaner on theicon selection window 630. The user interface unit displays arecommended instruction list on the basis of the operation state of therobot cleaner.

When an instruction “stop” is selected from the recommended instructionlist, the user interface unit displays “stop” on the conversation inputwindow. When the send button is touched, the user interface unitdisplays the device names and instructions on the conversation displaywindow 610.

In this case, when the robot cleaner is determined to be in a chargingoperation, the control unit of the control device checks an executableoperation, checks an instruction for controlling the checked operation,and displays a start instruction instructing to start cleaning and astop instruction instructing to stop recharging in the recommendedinstruction list.

As another example, when the air conditioner is being stopped, thecontrol unit may display an operation start command as the recommendedinstruction. When the air conditioner is in operation, the control unitmay display an operation stop instruction, a target temperature up/downinstruction, and an operation mode change instruction.

As shown in FIG. 33, when the message displayed on the conversationinput window 620 is the device name, the user interface unit displays anicon of an electric device having the device name on the icon selectionwindow 630. When a preset shortcut letter is input through the textwindow 650, the user interface unit displays the shortcut letter on theconversation input window. When the send button is touched, the userinterface unit displays an instruction corresponding to the device nameand the shortcut letter on the conversation display window 610.

In this case, the control device stores an instruction for each shortcutletter. In this case, the instruction for each shortcut letter may beset differently depending on the electric device.

That is, the control unit of the control device checks an instructioncorresponding to the input shortcut letter and controls a display of thechecked instruction.

For example, when “Robot Cleaner” is input through the text window 650,the user interface unit displays a robot cleaner on the conversationinput window 620 and also displays an icon of the robot cleaner on theicon selection window 630. When “c” is input through the text window 650and then the send button is touched, the user interface unit displaysthe instruction “clean” having the shortcut letter of “c” on theconversation display window 610 in addition to the device name (RobotCleaner).

In this case, it is possible to easily chat with an electric device byinputting an instruction using the shortcut letter upon the chatting.

The invention claimed is:
 1. A control device for controlling anoperation of at least one electric device, the control devicecomprising: a display configured to display a user interface comprising:a chatting screen, the chatting screen comprising a conversation displaywindow for displaying a conversation with the at least one electricdevice, a conversation input window for receiving a character forcontrolling an operation of the at least one electric device, an iconselection window for displaying an icon corresponding to the at leastone electric device, and a send button for sending the characterreceived through the conversation input window; and at least oneprocessor configured to: check an electric device corresponding to acontrol instruction received through the conversation input window,control to display an icon, and send the control instruction to theelectric device corresponding to the displayed icon.
 2. The controldevice of claim 1, further comprising storage configured to store thecontrol instruction, wherein, the at least one processor is furtherconfigured to, when the control instruction is for simultaneouslycontrolling two or more electric devices: send the received character tothe two or more electric devices, and control to display an icon for allelectric devices in the icon selection window.
 3. The control device ofclaim 1, wherein, the at least one processor is further configured to,when the control instruction is received through the conversation inputwindow while an icon for all electric devices is displayed in the iconselection window, control to send the received control instruction to acontrollable electric device.
 4. The control device of claim 1, furthercomprising storage configured to store the control instruction, wherein,the at least one processor is further configured to, when the controlinstruction is for individually controlling any one electronic device:check an electric device corresponding to the control instruction,control an operation of the checked electric device corresponding to thecontrol instruction, and control to display an icon of the checkedelectric device in the icon selection window.
 5. The control device ofclaim 4, wherein, the at least one processor is further configured to:when two or more electric devices correspond to the control instructionfor individually controlling any one electronic device, display a listof the two or more electric devices, when any one is selected from amongthe two or more electric devices, control an operation of the selectedelectric device based on the control instruction, and control to displayinformation regarding the operation of the selected electric device inthe conversation display window of the chatting screen.
 6. The controldevice of claim 5, wherein the at least one processor is furtherconfigured to, when the two or more electric devices correspond to thecontrol instruction for individually controlling any one electronicdevice, control to display an icon for all electric devices in the iconselection window.
 7. The control device of claim 1, wherein the at leastone processor is further configured to, when the character receivedthrough the conversation input window is a device name, control todisplay an icon of an electric device having the device name in the iconselection window.
 8. The control device of claim 1, wherein the at leastone processor is further configured to: when the character receivedthrough the conversation input window is a shortcut letter, check aninstruction corresponding to the shortcut letter and control anoperation of an electronic device corresponding to the checkedinstruction, when the character received through the conversation inputwindow is not a shortcut letter, check an instruction having asimilarity equal to or greater than a certain value with respect to thereceived character and control to display the checked instruction, andwhen one character is received through the conversation input window,check an instruction having the received character as a first letter andcontrol to display the checked instruction.
 9. The control device ofclaim 1, wherein the at least one processor is further configured to:check an operation state of the electric device corresponding to thedisplayed icon, and control to display a list of executable recommendedinstructions based on the checked operation state.
 10. The controldevice of claim 1, wherein the at least one processor is furtherconfigured to control to output a list of instructions for controllingthe electric device corresponding to the displayed icon.
 11. A controlmethod of a control device that controls an operation of at least one ofa plurality of electric devices, the control method comprising: when acharacter is received through a conversation input window of a userinterface unit, determining whether the received character is a controlinstruction; when the received character is the control instruction,checking an electric device corresponding to the control instruction;displaying an icon of the checked electric device in an icon selectionwindow of the user interface unit; and sending the control instructionto the checked electric device.
 12. The control method of claim 11,further comprising: when the control instruction is for individuallycontrolling one electric device, checking an electric devicecorresponding to the control instruction and displaying an icon of thechecked electric device in the icon selection window; and when thecontrol instruction is for simultaneously controlling two or moreelectric devices, displaying an icon for all electric devices in theicon selection window.
 13. The control method of claim 12, furthercomprising: when the two or more electric devices correspond to thecontrol instruction for individually controlling one electronic device,performing control to display a list of the two or more electricdevices; when any one is selected from among the two or more electricdevices, controlling an operation of the selected electric device basedon the control instruction; and displaying information regarding theoperation of the selected electric device in the conversation displaywindow of the user interface unit.
 14. The control method of claim 11,further comprising, when the control instruction is received through theconversation input window while an icon for all electric devices isdisplayed in the icon selection window, sending the control instructionto the plurality of electric devices.
 15. The control method of claim11, further comprising: when the character received through theconversation input window is a device name, displaying an icon of anelectric device having the device name; when the character receivedthrough the conversation input window is a shortcut letter, displayingan instruction corresponding to the shortcut letter; when the characterreceived through the conversation input window is not an instruction,checking an instruction having a similarity equal to or greater than acertain value with respect to the received character and displaying thechecked instruction; and when one character is received through theconversation input window, checking an instruction having the receivedcharacter as a first letter and displaying the checked instruction. 16.The control method of claim 11, further comprising: checking anoperation state of the electric device corresponding to the displayedicon; and displaying a list of executable recommended instructions basedon the checked operation state.